The North Atlantic swordfish fishery, under the management of the International Commission for the Conservation of Atlantic Tuna (ICCAT), is undergoing a management strategy evaluation (MSE) process.
ICCAT describes MSE as:
a collaborative process between Scientists and decision-makers that involves using computer simulation to compare the relative ability to achieve a set of management objectives using alternative Management Strategies, defined as different combinations of data collection schemes, methods of analysis, harvest control rules and subsequent processes leading to management actions.
There are three main components in an MSE process:
The operating models, candidate management procedures, and performance metrics are developed as a collaborative effort between scientists, decision-makers, and other stakeholders in the fishery.
This document describes the specifications for the OMs, CMPs, and PMs that have been proposed and developed for the North Atlantic swordfish (hereafter swordfish) fishery.
It is a living document and will be continued to be updated so that it reflects the current state of the swordfish MSE process.
Members of the Swordfish Species Group (hereafter the Group) are encouraged to provide feedback, comments, or edits to any part of this document.
The document is written using the Markdown format and can edited in any text editor. The source document is available on the ICCAT/nswo-mse GitHub repository.
Group members can make edits to the document either directly in the online repository or by cloning the repository and submitting pull requests with their edits. Alternatively, they can email questions or comments to Adrian. The former approach has the advantage that all comments, questions, and edits are immediately visible to all members of the Group.
Group members can also use the Discussions feature on the Github repository to post questions, comments, or points for discussion related to any aspect of this document or the MSE process in general.
This document is available at the North Atlantic Swordfish MSE homepage.
The SWOMSE R package has been developed to conduct the MSE for the Atlantic swordfish fishery. All code used for the MSE is open-source and reproducible. Group members can install the package and reproduce the analysis on their own machines.
The SWOMSE package used for the swordfish MSE uses the openMSE framework. The openMSE package is automatically installed when SWOMSE is installed, and all openMSE functions are available to the user when the SWOMSE package is loaded.
openMSE is an R package that has been developed for conducting fast, flexible, and transparent, MSE for a wide range of fisheries. openMSE is an umbrella package that includes the MSEtool, SAMtool and DLMtool packages. A non-technical description of openMSE and its key features is available on the openMSE website.
The operating model in openMSE, including assumptions and equations, is described in detail in Carruthers & Hordyk (2018).
The SWOMSE User Manual includes instructions for installing the SWOMSE package and running the MSE analyses.
Previously, the swordfish operating models were based on the 2017 stock assessment (Anon., 2017) using Stock Synthesis 3 (SS3, Methot & Wetzel, 2013).
A new stock assessment was conducted in 2022. The swordfish operating models have been updated based on this new assessment.
The report for the 2022 stock assessment is published in SCRS/2022/012. Details of the 2022 assessment are published in SCRS/2022/124. The data used in the 2022 assessment, and the structure and assumptions of the assessment model are summarized in the sub-sections below.
The assessment used landings data from 8 longline fleets (7 national fleets and ‘Other’ representing all catches not accounted for in the national fleets) (Table 3.1 and Figure 3.1).
There were 6 indices of abundance derived from catch-per-unit-effort (CPUE) data from the national fleets (Table 3.1 and Figure 3.1). Data from the Japan longline fleet was split into two phases: Early (1950 - 1993), and Late (1994 - 2020) due to changes in fishing practices in these periods. Similarly, the Chinese-Taipei longline fleet was split into two phases: Early (1968 - 1989) and Late (1997 - 2020). Five additional survey indices were developed from the age-specific (ages estimated from length data) CPUE from the Spain longline fleet (Table 3.1 and Figure 3.1).
Length composition data from 7 national longline fleets (Spain, U.S., Canada, Japan (Early, Late), Portugal, Chinese-Tapai (Early, Late), and Morocco) and the Canada/USA harpoon fleet (Figure 3.1). Mean weight data from two fleets (U.S. and Canada) were also used (Figure 3.1).
The catchability coefficient (q) for the CPUE indices for Canada, Morocco, and the Age-1, Age-2, and Age-4 Spain age-specific survey indices, was made a function of the Atlantic Multidecadal Oscillation (AMO) (see SCRS/2022/124 for details).
The effective sample size (ESS) for the length composition data was established by adjusting ESS until unity was reached between modeled ESS and the Francis suggested sample size (See Anon., 2017 for details).
| Name | Code |
|---|---|
| EU Spain longline (LL) | SPN 1 |
| USA LL | US 2 |
| Canada LL | CAN 3 |
| Japan LL Early | JPN ERLY 4 |
| Japan LL Late | JPN LATE 5 |
| EU Portugal LL | PORT 6 |
| Chinese Taipai LL Early | CHT EARLY 7 |
| Chinese Taipai LL Late | CHT LATE 8 |
| Morocco LL | MOR 9 |
| Canada USA Harpoon | HRPN 10 |
| Other LL by the other CPCs, and all other gears except HP | OTH 11 |
| USA Survey | US Survey 12 |
| EU Portugal Survey | PORT Survey 13 |
| Age 1 Survey | Age 1 |
| Age 2 Survey | Age 2 |
| Age 3 Survey | Age 3 |
| Age 4 Survey | Age 4 |
| Age 5 plus Survey | Age 5 plus |
Figure 3.1: Summary of the data used in the 2022 stock assessment.
The SS3 model used one season, one area, and two sexes.
Natural mortality for both male and female was fixed at 0.2 for all age classes. Maturity-at-age was knife-edge, with 50% at age-5 and 100% thereafter. Fecundity was proportional to body weight. The growth parameters for the 2022 assessment were fixed at same used in the 2017 assessment, which were developed during the 2017 ICCAT Swordfish Data Preparatory Meeting (See Anon., 2017 for details).
Selectivity was modeled as a function of length. Dome-shaped selectivity was allowed for five fleets: EU-Spain, USA, Japan, EU-Portugal, and Morocco. Asymptotic selectivity was assumed for Canada, Chinese-Taipei and Other. The age-specific survey CPUEs were modeled with a fixed age-based selectivity.
The 2022 assessment assumed a minimum legal length of 119 cm lower jaw fork length (LFFL) for all fleets from 1993 - 2020, and estimated the selectivity and retention curves from the available data. Discard mortality was either estimated from the observer data (USA and Canada) or fixed at values taken from the literature (see SCRS/2022/124).
Expected recruitment to age-0 was calculated from the total spawning stock biomass using the Beverton-Holt stock-recruit function. The standard error for the log-normally distributed recruitment deviations (sigmaR) was fixed to 0.2. Steepness (h) was fixed at the value estimated in the 2017 assessment (0.88).
The 2022 stock assessment was used as the Base Case model for developing the operating models (OMs) evaluated in the MSE.
In 2019, the Swordfish Species Group developed an OM uncertainty grid using a full factorial design with 6 axes of uncertainty:
Natural mortality (M) - three levels: 0.1, 0.2, 0.3
Recruitment variability (sigmaR; \(\sigma_R\)) - two levels: 0.2, 0.6
Steepness (h) - three levels: 0.6, 0.75, 0.90
CPUE Lambda - three levels: 0.05, 1, 20
Directional trend in catchability (llq) - two levels: TRUE, FALSE
Environmental covariate (env) - two levels: TRUE, FALSE
The directional increase in catchability was modeled by assuming a 1% average annual increase in catchability throughout the history of the fishery, and adjusting the CPUE indices accordingly.
The environmental covariate was included by modeling catchability as a function of the AMO, as was done in the 2017 and 2022 stock assessments.
This factorial design of the 6 axes of uncertainty, each with 2 or 3 levels, resulted in an uncertainty grid of 216 OMs.
Previous analyses of the OM uncertainty grid based on the 2017 assessment revealed that the three levels of natural mortality and steepness had the largest impact on the predicted stock dynamics Hordyk et al., 2021 and are therefore the most important axes of uncertainty in the OM grid.
The second level of recruitment variability (\(\sigma_R = 0.6\)) had a minor impact on the predicted stock dynamics Hordyk et al., 2021, but did influence the relative performance of candidate management procedures Hordyk, 2021. This second level is now treated as a robustness test (see below for more details).
The fourth axis of uncertainty was intended to evaluate the effect of alternative relative weightings of the length composition data and the indices of abundance. The three levels reflect a complete down-weighting of the indices of abundance (0.05; effectively only fitting the model to the length composition data), leaving the relative weighting of the two data sources unchanged from that used in the assessment (1), and up-weighting the indices of abundance so the model ignores the length composition data (20). This was done because of apparent conflicting signals between the length composition data and some of the indices of abundance, and the high computation demand of conducting the recommended iterative re-weighting procedure across all OMs in the grid (Francis, 2011). However, this iterative re-weighting procedure has now been conducted for the new operating models based on the 2022 assessment, and therefore this axis of uncertainty has been modified to two levels: 1) fit the assessment to both length and CPUE data and conduct the iterative re-weighting procedure, and 2) only fit the model to the CPUE data. This second level is now treated as a robustness test (see below for more details).
The fifth axis of uncertainty with the assumed 1% increase in catchability for the indices of abundance had a relatively minor influence on both the predicted stock dynamics Hordyk et al., 2021 and the performance of candidate management procedures Hordyk, 2021. Therefore it was decided that the default assumption for the operating models was to use the same indices as were used in the 2022 stock assessment; .e., the indices were not adjusted for an assumed 1% annual increase in changeability. The second level of this axis, where the indices are adjusted for an assumed increase in catchability, is now treated as a robustness test (see below for more details).
In both the 2017 and 2022 stock assessments, the catchability coefficient (q) for the CPUE indices for Canada, Japan, Portugal, Morocco, and the Spain age-specific survey indices, was made a function of the Atlantic Multidecadal Oscillation (AMO). Including this environmental covariate resulted in a better statistical fit to the data for these indices. The sixth axis of uncertainty examined the impact of not including this environmental covariate in the stock assessment. The analyses revealed that removing the environmental covariate from the assessment model had no detectable influence on either the predicted stock dynamics Hordyk et al., 2021 or the performance of candidate management procedures Hordyk, 2021. Therefore, the environmental covariate was included in all models in the OM grid. Further examination of the impact of changing environmental conditions on the performance of the candidate management procedures may be examined in additional robustness tests (see below for more details).
This same pattern in results was found when the analyses were re-conducted with the new OM grid based on the 2022 assessment.
The OMs, conditioned using SS3, were imported into MSE framework using the SS2MOM function in MSEtool (see here for details on the function).
Previously, the 2-sex SS3 models were imported into a combined single-sex operating model in the MSE framework. The MSE framework was updated in 2022, and the imported operating models now maintain the separate sexes in the same manner as the SS3 assessment. Since the fishery is managed with a single overall total allowable catch (TAC), the individual fishing fleets were aggregated together in the operating models.
The 2-sex operating model are available in the SWOMSE package as objects of class MOM (multi-stock operating model). For example, the 2022 stock assessment is available as the base case operating model named MOM_000:
library(SWOMSE)
MOM_000@Name## [1] "M:0.2 sigmaR:0.2 steepness:0.88 Include CAL:TRUE llq:1 env:1 Class:Base Case"The operating models have been classified into two categories: Reference OMs and Robustness OMs. Table 5.1 provides a general overview of the operating models includes in the MSE analysis. The following sub-sections describe the Reference and Robustness OMs in more detail.
| Class | Purpose | Description | OM.objects |
|---|---|---|---|
| Base Case | The 2022 Stock Assessment, considered the Base Case OM | 2022 Stock Assessment | MOM_000 |
| Reference | Span the key plausible uncertainties in natural mortality (M) and steepness of the Beverton Holt stock recruit relationship (h) | Same as Base Case with 3 levels of M (0.1, 0.2, 0.3) and 3 levels of h (0.69, 0.80, 0.88) | MOM_001, MOM_002, MOM_003, MOM_004, MOM_005, MOM_006, MOM_007, MOM_008, MOM_009 |
| R1. Lower Steepness | Evaluate sensitivity to stock with low resilence | Same as Base Case with 3 levels of M (0.1, 0.2, 0.3) and 1 level of h (0.6) | |
| R2. Higher sigmaR | Evaluate sensitivity to higher variability in recruitment process error | Same as Reference, with higher recruitment variability | |
| R3. Remove CAL | Evaluate impact of only using indices of abundance in OM conditioning (i.e., don’t include catch at length data in the model fitting) | Same as Reference, except OMs only conditioned on indices of abundance (not the catch-at-length data) | |
| R4. Increase q | Evaluate impact of an historical increase catchability that was not accounted for in the standardization of the indices of abundance | Same as Reference, except an assumed 1 percent average annual increase in catchability during the historical period | |
| R5. Increasing q | Evaluate impact of catchability continuing to increase in projections | Same as R4. Increase q, except the 1 percent average annual increase in catchability is assumed to continue in the projections | |
| R6. Implementation Error | Evaluate impact of illegal, unreported, or unregulated catches | Same a Reference, except a consistent unreported 10 percent overage in catch relative to TAC | |
| R7. Climate Change Recruitment | Evaluate impact of systematic pattern in recruitment deviations in projection periods; a proxy for impact of climate change on productivity | Same as Reference, except recruitment deviations in projections have postive/negative trends | |
| R8. Size Limit | Evaluate impact of different size limits, including removing all size regulations | Same as Reference, except size limit fixed to 120 cm or removed completely | |
| R9. Alternative Management Cycles | Evaluate the impact of a longer management cycle | Same as Reference, except the management cycle is modified from the default 3 years |
The Base Case operating model is the 2022 Stock Assessment. See the Stock Assessment section for more details.
The Reference OMs are based on the Base Case Model and span the plausible range for the key uncertainties. The Candidate Management Procedures (CMPs) are tuned across the Reference OMs (see the CMP Tuning section for more details.
Based on the analyses described above, a set of 9 operating models were identified as the Reference OMs. These operating models spanned the three levels of M (0.1, 0.2, and 0.3) and three levels h (0.69, 0.80, and 0.88).
The values for steepness in the Reference Set were determined at the 2023 Intersessional Meeting of the Swordfish Working Group. The lower and upper values were set at the 95% confidence intervals of a prior distribution for the steepness parameter used in the 2022 assessment.
The center value was calculated by first converting steepness to the Goodyear compensation ratio (CR) with \(\text{CR}=\frac{4h}{1-h}\). Using this equation, the CR for h = 0.69 and h = 0.88 are 8.90 and 29.33 respectively. A mid-point between these two CR values is calculated as:
\(\text{CR}_\text{mid} = e^{\frac{\sum_i\log\text{CR}_i}{2}}=16.16\), where the lower and upper values are 16.16/1.815 and 16.16 x 1.815 respectively. Back-calculating to steepness with \(h=\frac{\text{CR}}{\text{CR}+4}\) gives \(h=0.80\).
Initially the OMs with the lower steepness value \((h=0.6)\) was also considered to be in the Reference OMs. However, recent analysis and discussion with the MSE Technical Team has revealed that a such a low steepness is unlikely for North Atlantic swordfish. Consequently, the OMs with \(h=0.6\) have been re-classified into a Robustness Set.
The Reference OMs had the following assumptions:
Robustness set 1 (R1. Lower Steepness) evaluates the performance of the CMPs under conditions where the stock has lower than expected resilience. These 3 OMs have the same assumptions as the Base Case, except that steepness (h) is fixed to 0.6, and natural mortality has three levels: 0.1, 0.2, and 0.3.
This set of 6 operating models has the same structure and assumptions as the Reference Set, with the exception that the recruitment variability was assumed to higher \((\sigma_R = 0.6)\)
This set of 6 operating models has the same structure and assumptions as the Reference Set, with the exception that the model was only fit to the indices of abundance (i.e., fits to the length composition data were not included in the total likelihood).
This set of 6 operating models had the same structure and assumptions as the Reference Set, with the exception that the indices of abundance were modified to assume an average 1% increase in catchability over the historical period.
This set of 6 operating models has the same assumptions as R4. Increase q, except that the 1% average annual increase in catchability is assumed to continue in the future. The implementation of this assumption is made in during the forward projections by passing a custom function to the CMPs which adjusts the simulated indices of abundance assuming a 1% average annual increase in catchability.
This set of 6 operating models has the same assumptions as the Reference OMs, except that there is a consistent non-reported 10% overage in the TAC in each year. This test is designed to replicate illegal, unreported, or unregulated (IUU) fishing where the actual removals are 10% higher than the TAC and reported catches.
This set of OMs are based on the Reference OMs and includes several scenarios to evaluate the impact of climate change on the performance of the CMPs. These scenarios are designed to simulate the impact of climate change on the productivity of the stock, and are implemented in the recruitment deviations for the projection years.
Figure 5.1 shows the Base Case (no trend in recruitment deviations) and the three scenarios:
Decreasing Trend: a linear decrease in recruitment strength in imposed over the recruitment deviations that are generated for the projection years. The recruitment deviations are assumed to decrease by 20% at the end of the projection period.
Increasing Trend: a linear increase in recruitment strength in imposed over the recruitment deviations that are generated for the projection years. The recruitment deviations are assumed to increase by 20% at the end of the projection period.
Increased Variability: additional variability is added to the future recruitment by sampling between the upper and lower bounds shown in the plot.
Figure 5.1: The Base Case and 3 scenarios for persistent patterns in recruitment deviations in the projection years.
This set of 6 operating models has the same assumptions as the Reference OMs with the exception that the size limit is modified in the projection period. This change is implemented in the CMPs and includes two scenarios:
Fixed 120 cm Size Limit: retention is assumed knife-edge at 120 cm for all fleets, with discard mortality applied to the fish that are captured and released below this size.
No Size Limit: the size limit is removed and all fish that are captured are retained and contribute to the TAC.
These two scenarios are compared to the Reference OMs, which are designed to replicate the current size regulations (see above for more details).
This set of 6 operating models has the same assumptions as the Reference OMs with the exception that the management cycle is modified from the default assumption of 3 years (see below) to a 4-year interval.
Note: The operating models have been updated and re-conditioned in May 2023. The OM Summary Report and individual diagnostic reports below currently refer to an earlier set of operating models. The reports will be updated in the future.
A Summary Report summarizes the diagnostic checks, the calculated biological reference points, and the estimated stock status relative to those reference points, across the Reference and Robustness operating models.
Individual diagnostic reports with objective function values and plots of model fits and patterns in residuals are available for each of the Reference and Robustness OMs (and the 2022 assessment) on the North Atlantic Swordfish MSE homepage.
The historical spool-up period was generated by importing the output from the SS3 assessment models and running the MSE simulation framework to recreate the historical fishery dynamics. No additional uncertainty was added to the historical simulations; that is, all \(50\) simulations were identical (e.g., same recruitment deviations) for the historical spool-up period.
The operating models have been conditioned on data up to and including 2020 (see the Data section), and therefore the projection period for the MSE framework starts in the following year, i.e., 2021. Currently, model uses assumed catches for the first three years (2021 - 2023) and then implements the management procedures in the following year (i.e., 2024). See the Future Catches section for more information.
The observed fishery data was imported into the MSE framework and made available to the CMPs. Three sources of data available to the CMPs: 1) catch data, 2) a primary index of abundance, and 3) additional individual indices.
The catch data used in the OM conditioning are made available in the simulated data that is provided to the CMPs (Figure 7.1).
Figure 7.1: The historical total catch and total allowable catch (TAC) for the North Atlantic swordfish.
At the 2020 Swordfish MSE technical meeting (4 – 5 June 2020), the Group chose to use the Combined Index (Ortiz et al., 2017) as the primary index for the development of CMPs. This index was updated for the 2022 assessment. The Combined Index was imported into the SWOMSE framework and made available to the CMPs (Figure 7.2).
Figure 7.2: The Combined Index for the North Atlantic swordfish fishery.
The additional individual fishery-dependent and survey indices are also imported into the MSE framework and made available to the CMPs (Figure 7.3).
Figure 7.3: The additional individual fishery-dependent and survey indices for the North Atlantic swordfish fishery.
The current simulation specifications for the MSE are:
The stock-recruitment relationship in the projection years was modeled using the Beverton-Holt function, with steepness fixed at the value assumed in the OM conditioning.
Recruitment deviations for the projection period were generated assuming a log-normal distribution with mean \(\mu_R\) and variance \(\sigma_R^2\), calculated as:
\[\mu_R = -0.5\sigma_R^2\left(1-\frac{\text{AC}}{\sqrt{(1-\text{AC}^2)}}\right)\]
where AC is the lag-1 autocorrelation factor calculated from the historical recruitment deviations and \(\sigma_R^2\) is the variance of the recruitment deviations specified in the OM (i.e., sigmaR^2 defined above).
The recruitment deviations for the projection period were generated independently for each simulation.
Biological parameters such as mean weight-at-age and maturity-at-age were constant for all years (historical and projection).
Currently, the selectivity-at-length for the projection years is fixed to the estimated selectivity pattern from the last historical year for each OM. The reviewer of the SWOMSE R code highlighted this as a possible uncertainty and recommended that the Group consider testing alternative assumptions for the selectivity pattern in the projection years (Anon., 2021).
The Working Group will discuss this assumption in more detail and decide if it wishes to run scenarios with alternative assumptions for the selectivity pattern in the projection years.
No time-varying catchability is assumed for the projection years.
The total allowable catch (TAC) recommended by the CMPs is assumed to be implemented without error. That is, annual catches are equal to the TAC recommendations (provided sufficient vulnerable biomass was available). The Group determined that this assumption was appropriate based on the fact that historically the total landed catch has been below the total allowable catch (TAC) (see Figure 7.1).
A Robustness Test (R6. Implementation Error) has been designed to evaluate the consequences of violating this assumption.
The operating models also assume that the TACs recommended by the CMPs apply to the landed catch and do not include discards; that is, the actual removals will be higher than the TACs if there is discard mortality on the sub-legal fish.
The CMPs are first implemented in 2024, and every 3rd year after that, with the TAC held constant in the interim years. Since the MSE projection period begins in 2021 (the year after the conditioning), the landed catches for the first few projection years until the CMPS are implemented (2021, 2022, and 2023) are fixed at assumed values (Table 8.1).
| Year | Catch | Details |
|---|---|---|
| 2021 | 9729 | Reported Catch |
| 2022 | 13200 | Assumed Catch (equal to TAC) |
| 2023 | 13200 | Assumed Catch (equal to TAC) |
The primary index of abundance (Combined Index) was generated in the projection years by adding observation error to the projected stock biomass, and standardizing to be on the same scale as the historical observed Combined Index.
The additional individual indices were generated in the projection years in the same manner, using the simulated stock biomass (or abundance in some cases).
The observation error was generated by calculating the statistical properties of the residuals from the fit of each respective index to the simulated stock (biomass or abundance) during the historical period. The procedure for calculating the residuals for the projection years is described in detail in the openMSE documentation.
It was decided at the 2023 Intersessional Meeting of the Swordfish Working Group to calculate the statistical properties of the observation error for the Combined Index for the years 1999 – 2020. This period was chosen as it includes only the years where the data used to generate the index included the majority of the fleets.
Following the assumptions of the stock assessment, it is assumed that all indices are proportional to the respective stock biomass or abundance (i.e., it is assumed that the indices are not hyper-stable or hyper-deplete). The only exception to this is Robustness Set 5 where the indices are assumed to have a 1% annual average increase in catchability.
Following the assumption of the stock assessment, the observed catches in the projection years were generated with no observation error. That is, the observed catches in the MSE framework are equal to the actual simulated landed catches (provided there is sufficient biomass in the simulated population to support the prescribed TAC). The model does not include a bio-economic component and inter-annual change in fishing effort is unconstrained.
In the MSE framework, the CMPs are always provided with the simulated fishery data up to the previous year. That is, if a CMP is implemented in year \(t\), the catch and indices of abundance data provided to the CMP are up to and including year \(t-1\).
Data lags are then handled in the CMP code, where the default assumption is that the catch and index data are lagged by 2 years from the year where the TAC will be implemented; e.g., for setting the TAC for 2024, the data is up to and including 2022. See the CMP (see the CMP Development Guide for more details).
At the 2021 Intersessional meeting, the Group proposed a 3-year MP advice interval (see Table 5 in the Report of the 2021 ICCAT Intersessional Meeting of the Swordfish Species Group).
The operating models have been conditioned on data up to 2020. As described above, the catches in 2021 – 2023 are fixed to assumed values and the CMPs are first applied in 2024.
The 3-yr management interval is applied within the CMP code (see the CMP Development Guide for more details).
Robustness Set 9 (R9) evaluates the impact of alternative management intervals.
25 performance metrics (PMs) have been developed for the North Atlantic Swordfish MSE (Table 9.1). These PMs are grouped into four families:
Status: the probability the stock is in the green quadrant of the Kobe matrix
Safety: the probability of the stock not falling below the biological limit reference point
Yield: the TAC in the projection years (ton)
Stability: the variation in the TAC between management cycles
The performance metrics are calculated over all the simulations (50) in each operating model. For example, PGK_short is calculated as the proportion of the 50 simulations where the spawning biomass (\(SB\)) is above \(SB_{MSY}\) and fishing mortality (\(F\)) is below \(F_{MSY}\).
The performance metrics can also be calculated over several operating models, for example the entire Reference Set. This is done by first combining the MSE results for each OM (e.g., the 9 Reference OMs) into a single set of results (in this example, resulting in 450 simulations (9 OMs multiplied by 50 simulations per OM)), and then calculating the performance metrics over all results.
The Minimum Acceptable Performance values listed in Table 9.1 will be used as alternative values to determine if candidate management procedures meet the minimum performance criteria.
More details on the individual performance metrics are in the sections below.
| Family | Name | Description | Minimum Acceptable Values |
|---|---|---|---|
| Status | PGK_short | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 1-10 (2024-2033) | 51, 60, 70 |
| PGK_med | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 11-20 (2034-2043) | 51, 60, 70 | |
| PGK_long | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 21-30 (2044-2053) | 51, 60, 70 | |
| PGK | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) over all years (2024-2053) | 51, 60, 70 | |
| PGK_30 | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in year 30 (2053) | 51, 60, 70 | |
| POF | Probability of Overfishing (F>FMSY) over all years (2024-2053) | ||
| PNOF | Probability of Not Overfishing (F<FMSY) over all years (2024-2053) | ||
| Safety | LRP_short | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of the first 10 years (2024-2033) | 5, 10, 15 |
| LRP_med | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of years 11-20 (2034-2043) | 5, 10, 15 | |
| LRP_long | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of years 21-30 (2044-2053) | 5, 10, 15 | |
| LRP | Probability of breaching the limit reference point (SB<0.4SBMSY) in any year (2024-2053) | 5, 10, 15 | |
| Yield | TAC1 | TAC (t) in the first implementation year (2024) | |
| AvTAC_short | Median TAC (t) over years 1-10 (2024-2033) | ||
| AvTAC_med | Median TAC (t) over years 11-20 (2034-2043) | ||
| AvTAC_long | Median TAC (t) over years 21-30 (2044-2053) | ||
| Stability | VarC | Median variation in TAC (%) between management cycles over all years | |
| MaxVarC | Maximum variation in TAC (%) between management cycles over all years | No minimum value and 25 |
The following sections show figures and statistics for examples from each family of Performance Metrics. These results include two example candidate management procedures (MP1, MP2) (see Candidate Management Procedures), with a single operating model that includes 10 stochastic simulations.
Table 9.2 shows the summary performance statistics for the example CMPs. More details for each individual performance metric are in the sub-sections below.
| Family | Name | Description | MP1 | MP2 |
|---|---|---|---|---|
| Status | PGK_short | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 1-10 (2024-2033) | 0.29 | 0.11 |
| PGK_med | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 11-20 (2034-2043) | 0.77 | 0.34 | |
| PGK_long | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 21-30 (2044-2053) | 1.00 | 0.81 | |
| PGK | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) over all years (2024-2053) | 0.69 | 0.42 | |
| PGK_30 | Probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in year 30 (2053) | 1.00 | 1.00 | |
| POF | Probability of Overfishing (F>FMSY) over all years (2024-2053) | 0.16 | 0.39 | |
| Safety | LRP_short | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of the first 10 years (2024-2033) | 0.10 | 0.10 |
| LRP_med | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of years 11-20 (2034-2043) | 0.10 | 0.20 | |
| LRP_long | Probability of breaching the limit reference point (SB<0.4SBMSY) in any of years 21-30 (2044-2053) | 0.00 | 0.00 | |
| LRP | Probability of breaching the limit reference point (SB<0.4SBMSY) in any year (2024-2053) | 0.10 | 0.20 | |
| Yield | TAC1 | TAC (t) in the first implementation year (2024) | 13809.46 | 15492.82 |
| AvTAC_short | Median TAC (t) over years 1-10 (2024-2033) | 10764.90 | 12375.00 | |
| AvTAC_med | Median TAC (t) over years 11-20 (2034-2043) | 6652.16 | 8114.54 | |
| AvTAC_long | Median TAC (t) over years 21-30 (2044-2053) | 5020.85 | 7761.06 | |
| Stability | VarC | Median variation in TAC (%) between management cycles over all years | 0.21 | 0.19 |
| MaxVarC | Maximum variation in TAC (%) between management cycles over all years | 0.25 | 0.25 |
PGK_short calculates the probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in years 1-10 (2024-2033). This is calculated as the proportion of the total number of simulations that are in the Green Zone of Kobe Space in those years.
Figure 9.1 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for PGK_short in this example are: 0.29, 0.11.
Figure 9.1: An example of time-series plots of F/FMSY and SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>SBMSY and F<FMSY in 2033 are indicated with a green point, and those outside of the green space in the Kobe plot are indicated in red. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
PGK_med calculates the probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) over years 11–20 (2034–2043). This is calculated as the proportion of the total points (number of simulations x number of years) that are in the Green Zone of Kobe Space.
Figure 9.3 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for PGK_med in this example are: 0.77, 0.34.
Figure 9.3: An example of time-series plots of F/FMSY and SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>SBMSY and F<FMSY in years 2029–2033 are indicated with a green point, and those outside of the green space in the Kobe plot are indicated in red. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
PGK_long calculates the probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) over years 21–30 (2044–2053). This is calculated as the proportion of the total points (number of simulations x number of years) that are in the Green Zone of Kobe Space.
Figure 9.4 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for PGK_long in this example are: 1, 0.81.
Figure 9.4: An example of time-series plots of F/FMSY and SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>SBMSY and F<FMSY in years 2034–2053 are indicated with a green point, and those outside of the green space in the Kobe plot are indicated in red. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
PGK calculates the probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) over all years (2024–2053). This is calculated as the proportion of the total points (number of simulations x number of years) that are in the Green Zone of Kobe Space.
Figure 9.5 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for PGK in this example are: 0.69, 0.42.
Figure 9.5: An example of time-series plots of F/FMSY and SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>SBMSY and F<FMSY are indicated with a green point, and those outside of the green space in the Kobe plot are indicated in red. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
PGK_30 calculates the probability of being in Green Zone of Kobe Space (SB>SBMSY & F<FMSY) in year (2053). This is calculated as the proportion of the total number of simulations that are in the Green Zone of Kobe Space.
Figure 9.6 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for PGK_30 in this example are: 1, 1.
Figure 9.6: An example of time-series plots of F/FMSY and SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>SBMSY and F<FMSY in 2053 are indicated with a green point, and those outside of the green space in the Kobe plot are indicated in red. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
POF calculates the probability of over-fishing (F>FMSY) over all years (2024–2053). This is calculated as the proportion of the total number of points (number of simulations x number of years) where F is greater than FMSY.
Figure 9.7 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for POF in this example are: 0.16, 0.39.
PNOF reports the probability of not overfishing, calculated as \(1-\text{POF}\), i.e. 0.84, 0.61.
Figure 9.7: An example of a time-series plot of F/FMSY for a two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where F<FMSY are indicated with a green point, and red points indicate F>FMSY. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
LRP_short calculates the probability of breaching the limit reference point (LRP; SB<0.4SBMSY) in any of the first 10 years (2024-2033).
This is calculated as the proportion of simulations where SB<0.4SBMSY in any of the first 10 years.
Figure 9.8 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for LRP_short in this example are: 0.1, 0.1.
Figure 9.8: An example of a time-series plot of SB/SBMSY for a two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>0.4SBMSY are indicated with a green point, and red points indicate SB<0.4SBMSY. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
LRP_med calculates the probability of breaching the limit reference point (LRP; SB<0.4SBMSY) in any of years 11-20 (2034-2043).
This is calculated as the proportion of simulations where SB<0.4SBMSY in any of years 11-20.
Figure 9.9 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for LRP_med in this example are: 0.1, 0.2.
Figure 9.9: An example of a time-series plot of SB/SBMSY for a two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>0.4SBMSY are indicated with a green point, and red points indicate SB<0.4SBMSY. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
LRP_long calculates the probability of breaching the limit reference point (LRP; SB<0.4SBMSY) in any of years 21-30 (2044-2053).
This is calculated as the proportion of simulations where SB<0.4SBMSY in any of the last 10 years.
Figure 9.10 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The value for LRP_long in this example are: 0, 0.
Figure 9.10: An example of a time-series plot of SB/SBMSY for two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>0.4SBMSY are indicated with a green point, and red points indicate SB<0.4SBMSY. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
LRP calculates the probability of breaching the limit reference point (LRP; SB<0.4SBMSY) in any of the projection years (2024-2053).
This is calculated as the proportion of simulations where SB<0.4SBMSY in any year.
Figure 9.11 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The value for LRP in this example are: 0.1, 0.2.
Figure 9.11: An example of a time-series plot of SB/SBMSY for a two CMPs. This OM has 10 simulations, shown as the colored lines. Simulations where SB>0.4SBMSY are indicated with a green point, and red points indicate SB<0.4SBMSY. The Kobe plot on the bottom shows these points in the Kobe space. The numbers in the corners indicate the percent of points in each area of the Kobe space.
TAC1 calculates the median TAC (t) in the first year (2024). This is calculated as the median across the simulations of the 2024 TAC.
Figure 9.12 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for TAC1 in this example are: 13809, 15493.
Figure 9.12: An example of time-series plots of the TAC for two CMPs. This OM has 10 simulations, shown as the colored lines. The points indicate the values used to calculate the median value.
AvTAC_short calculates the median TAC (t) in the first 10 years (2024–2033). This is calculated as the median TAC across the simulations and years.
Figure 9.13 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for AvTAC_short in this example are: 10765, 12375.
Figure 9.13: An example of time-series plots of the TAC for a two CMPs. This OM has 10 simulations, shown as the colored lines. The points indicate the values used to calculate the median value.
AvTAC_med calculates the median TAC (t) in years 11 – 20 (2034–2043). This is calculated as the median TAC across the simulations and years.
Figure 9.14 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for AvTAC_med in this example are: 6652, 8115.
Figure 9.14: An example of time-series plots of the TAC for a two CMPs. This OM has 10 simulations, shown as the colored lines. The points indicate the values used to calculate the median value.
AvTAC_long calculates the median TAC (t) in the last 10 years (2044–2053). This is calculated as the median TAC across the simulations and years.
Figure 9.15 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for AvTAC_long in this example are: 5021, 7761.
Figure 9.15: An example of time-series plots of the TAC for two CMPs. This OM has 10 simulations, shown as the colored lines. The points indicate the values used to calculate the median value.
VarC calculates the median absolute change in the TAC between management cycles. MaxVarC reports the maximum change in TAC across all years and simulations.
These values are calculated as the absolute relative change in the TAC from one management cycle to the next.
Figure 9.16 shows an example of two CMPs (MP1, MP2) for an operating model with 10 simulations. The values for VarC in this example are: 0.21, 0.19. The value for MaxVarC in this example are: 0.25, 0.25.
Figure 9.16: An example of time-series plots of the relative change in the TAC between management cycles for two CMPs. This OM has 10 simulations, shown as the colored lines. The median and maximum absolute change are shown in the top left corner.
During development, the Candidate Management Procedures are tuned to achieve specific performance outcomes. These performance outcomes are calculated across all the Reference OMs; i.e, the MSE results from each Reference OM are aggregated together. 3 different tuning targets have been specified (Table 10.1).
There will be 3 versions of each CMP corresponding to metrics and targets described in Table 10.1. The names of the CMPs include the tuning code (i.e., ‘a’, ‘b’).
| Code | Metric | Target |
|---|---|---|
| a | PGK_short | 0.51 |
| b | PGK_short | 0.60 |
| c | PGK_short | 0.70 |
TODO - describe Slick